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Abstract 

Background: The quality of care delivered and clinical outcomes of care are of paramount importance. Wide 
variations in the outcome of emergency care have been suggested, but the scale of variation, and the way in 
which outcomes are inter-related are poorly defined and are critical to understand how best to improve services. 
This study quantifies the scale of variation in three outcomes for a contemporary cohort of patients undergoing 
emergency medical and surgical admissions. The way in which the outcomes of different diagnoses relate to each 
other is investigated. 

Methods: A retrospective study using the English Hospital Episode Statistics 2005-2010 with one-year follow-up for 
all patients with one of 20 of the commonest and highest-risk emergency medical or surgical conditions. The 
primary outcome was in-hospital all-cause risk-standardised mortality rate (in-RSMR). Secondary outcomes were 
1-year all-cause risk-standardised mortality rate (1 yr-RSMR) and 28-day all-cause emergency readmission rate (RSRR). 

Results: 2,406,709 adult patients underwent emergency medical or surgical admissions in the groups of interest. 
Clinically and statistically significant variations in outcome were observed between providers for all three 
outcomes (p < 0.001). For some diagnoses including heart failure, acute myocardial infarction, stroke and 
fractured neck of femur, more than 20% of hospitals lay above the upper 95% control limit and were statistical 
outliers. The risk-standardised outcomes within a given hospital for an individual diagnostic group were significantly 
associated with the aggregated outcome of the other clinical groups. 

Conclusions: Hospital-level risk-standardised outcomes for emergency admissions across a range of specialties vary 
considerably and cross traditional speciality boundaries. This suggests that global institutional infra-structure and processes 
of care influence outcomes. The implications are far reaching, both in terms of investigating performance at individual 
hospitals and in understanding how hospitals can learn from the best performers to improve outcomes. 
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Background 

Wide variations in clinical outcomes between hospitals 
have been described for a number of conditions. In 
addition, a number of high-profile failures of healthcare 
have been reported [1,2]. One common theme has been 
a failure to deliver safe and high quality care, with subse- 
quent poor clinical outcomes. In response to such varia- 
tions and failures, there have been suggestions that rating 
systems for hospitals could be developed in tandem with 
stringent hospital inspections. The aim of such a system 
would be to inform key stakeholders on the quality of care 
delivered by providers and to provide a method to prevent 
problems from developing, detect failure before harm is 
done and to facilitate a timely response. 

The feasibility of a single rating being able to achieve 
all these aims has been questioned by commentators, in- 
cluding a report by the Nuffield Trust [3,4]. In particular, 
a rating system can only be useful if significant varia- 
tions in outcome exist between hospitals, and can only 
be of value if the outcomes of different diagnoses are 
inter-related within individual hospitals [5]. Whether rat- 
ings should be limited to specific conditions, or whether 
they can be hospital-wide requires evaluation [6]. There- 
fore, it is important to define whether hospitals perform 
at a similar level of outcome across a range of condi- 
tions, or whether individual providers encompass a range 
of performance levels, dependent on specialty [7-9]. 

The objectives of this study were to determine whether 
the risk-standardised clinical outcomes for emergency 
admissions varied significantly between providers, and 
whether outcomes were inter-related between different 
diagnoses at a hospital level. 

Methods 

The data source for this retrospective study was the national 
administrative dataset, the Hospital Episode Statistics (HES) 
data, for the period 1 st April 2005 to 31 st March 2010. The 
HES detail every admission into the NHS and allow patients 
to be tracked between NHS hospitals and across years 
through the use of a unique pseudo-anonymised identifier. 
The dataset can be considered an inclusive record of 
National Health Service (NHS) hospital activity in England 
as there is a requirement for every hospital to submit a 
"minimum dataset" to the Department of Health. Each 
hospital submits data from its own Patient Administration 
System (PAS) to the centralised HES data warehouse 
which is then validated and cleaned by the NHS Informa- 
tion Centre according to a pre-specified list of rules. HES 
contains publicly available admitted patient care data from 
1989 onwards, with more than 12 million new records 
added each year. Each HES record contains a significant 
amount of information about the associated admission in- 
cluding patient demographics, the treating hospital, diag- 
nostic and procedural coding, length of stay, discharge 



status and destination. Emergency and elective admissions 
can be differentiated by specific combinations of codes re- 
lating to type of admission and diagnostic/procedural code 
sub-groups. As a result, it is possible to identify patient co- 
horts with specific characteristics with a high degree of ac- 
curacy [10]. A recent report based on HES data describes 
the approximate number of emergency admissions in 
England as more than 4.5 million per annum [11]. 

A five-year study period was chosen to achieve a prag- 
matic balance between sample size and the cohort reflect- 
ing contemporary practice. The HES were linked with the 
Office of National Statistics (ONS) Registry data to pro- 
vide longer-term and out-of-hospital mortality data. Clin- 
ical coding in the HES is through the OPCS-4 procedural 
coding system and ICD-10 diagnostic codes [12]. 

Twenty patient groups were used that covered the 
breadth of hospital emergency admissions in adults 
(>17years of age) to acute NHS (public) hospitals. The 
groups were chosen a-priori by quantifying the com- 
monest emergency admission diagnoses with the high- 
est numbers of deaths to all acute NHS hospitals using 
a sample of HES data (2010/2011) (Additional file 1: 
Tables SI and S2). ICD-10, OPCS-4 and HRG codes 
were used to create clinically meaningful groups. Medical 
conditions were defined by ICD-10 codes whilst surgical 
groups were defined by OPCS-4 codes. 

The primary outcome measure was in-hospital all-cause 
risk-standardised mortality rate (in-RSMR) and secondary 
outcomes were one-year all-cause risk- standardised mor- 
tality rate (lyr-RSMR) and 28-day all-cause emergency re- 
admission rate (RSRR) [13]. In-hospital mortality was 
defined as death occurring after admission to an index 
hospital and before discharge either from the index hos- 
pital or any subsequent receiving hospital in cases where 
the patient was transferred from the index hospital (a def- 
inition equivalent to a "super-spell" whereby a "spell" is 
HES -specific terminology denoting one continuous ad- 
mission at a hospital) [12,14]. RSRRs were quantified 
using only patients discharged alive from hospital. 

Established methods were used to extract and clean the 
data, and entries with missing key fields, such as operation 
dates, were excluded from analyses (representing 1.15% of 
the dataset) [15-17]. Each patient was included only once, 
using the diagnostic or procedural code in the primary field 
in the first occurrence for allocation to a clinical group. This 
prevented a single patient being included multiple times due 
to complications of care, or multiple surgical procedures. 

The confounding effect of inter-hospital transfers on 
in-RSMR and RSRR was accounted for as patients can 
be tracked between hospitals within the HES. Previous 
studies have shown that linking concurrent admissions 
together and assigning the ultimate outcome to the 
index hospital (super-spells) provides the most accurate 
reflection of in-hospital death rates [12]. 
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Statistical analyses 

The analyses had two distinct stages: first, to quantify 
the extent of variations in outcome within each clinical 
group; and second, to determine whether the outcomes 
of a specific clinical group were associated with the col- 
lective outcomes of the other clinical groups within the 
same hospital Statistical analysis was performed using 
SAS version 9.2 (SAS Institute, USA). 

Risk-standardisation was performed using hierarchical 
logistic regression models. Patient age, sex, RCS (Royal 
College of Surgeons) Charlson score, social deprivation 
index and year of admission formed first-level predictors 
[18]. Co-morbidity scores for cases were derived from 
the HES data by using published methodology [19]. The 
method relies on identification of pre-existing ICD-10 
diagnostic codes to denote co-morbidity. The score is 
created by calculating the number of co-morbidity cat- 
egories present in any admission episode in the preced- 
ing 365 days or in the index episode for any one case of 
interest. A case scoring in any one category (regardless 
of the number of times) achieves a score of 1, scoring in 
any two different categories achieves a score of 2 and a 
score in three or more categories achieves a score of 3. 
Cases without any flagged co-morbidities receive a score 
of 0. The scoring system is a modification (the Royal 
College of Surgeon s [RCS] modification) of the original 
score derived by Charlson [20,21]. The Index of Multiple 
Deprivation (IMD) overall ranking is made by combining 
seven weighted domains [22]. A score of 1 indicates the 
most deprived and 32482 the least deprived. This nu- 
merical range is sub-divided into fifths and cases are al- 
located into to one of the five quintiles according to 
their score (i.e. quintile 1 represents the most deprived 
cases and quintile 5 represents the least deprived cases). 

The second level of the RSMR and RSRR models per- 
mitted hospital-level random intercepts to vary in order 
to identify hospital-specific random effects and account 
for the clustering of patients within the same hospital 
[23]. This allowed, within- and between-hospital vari- 
ation to be separated, within the limits of the data, after 
adjusting for patient-level characteristics. Analysis was 
performed at NHS trust level (i.e. potentially including 
more than one physical site) and the term "hospital" is 
used synonymously with "trust". 

Variation in outcome 

Variations in outcome for each metric were assessed for 
each clinical group. Risk estimates from individual patient 
data were used to calculate the expected number of deaths 
for each condition at each hospital. For each clinical group, 
the discrepancy between the observed and expected mor- 
tality in each hospital was quantified. A statistically signifi- 
cant divergence was reported when it exceeded the 95% 



confidence interval of the Poisson distribution. Visually 
these were represented with risk-standardised funnel plots. 

Outcome inter-relationship analysis 

The risk-standardised outcome for each of the twenty clin- 
ical groups was compared against the risk-standardised ag- 
gregate outcome for the other clinical groups excluding the 
procedure/condition of interest (e.g. acute myocardial in- 
farction vs. an aggregate outcome of the other nineteen 
groups [i.e. excluding acute myocardial infarction]) using 
published methodology [8,14]. 

Hospitals were placed into quintiles containing equal 
numbers of patients based on their aggregate events 
rates. This was achieved by ranking hospitals according 
to their aggregate "other" event rate into five groups 
(quintiles) such that hospitals with the highest observed 
event rate relative to expectation (calculated by dividing 
the difference between the observed and expected num- 
bers by a measure of random variability) were in the 
highest numbered group (quintile 5) and those with the 
lowest observed event rate relative to expected were in 
quintile 1. Hospitals with very high event rates relative to 
expected would not necessarily be allocated to the highest 
quintile if the case load was small. Aggregation was per- 
formed using the providers mean Studentised residual ag- 
gregated across the remaining patient groups such that 
conditions with high event rates did not dominate the aggre- 
gation. Hospitals were assigned such that there were roughly 
equal numbers of patients in each quintile. Having assigned 
quintiles of aggregate "other" event rates, the combined 
event rate for the procedure/condition of interest was calcu- 
lated for each quintile (by multiplying the observed to ex- 
pected ratio by the crude total death rate for the procedure/ 
condition of interest). Results were expressed as a bar plot. 
The relationship between the outcome for each clinical 
group and the amalgamated outcome quintiles was tested 
using logistic regression. 

Reporting of the study complied with the STROBE 
guidelines [24]. 

Ethics committee approval 

The Wandsworth research ethics committee (WanREC) 
confirmed that no ethical approval was required for this 
study. 

Results 

2,406,709 admissions were included across 20 emergency 
groups in the five-year period. Summary demographic 
and outcome data for each group are provided (Table 1). 

Variations in outcomes 

Clinically important, and statistically significant, variations in 
outcome were seen across the range of medical and surgical 
conditions for in-RSMR, 1 yr-RSMR and RSRR (Table 2). 



Table 1 Demographic data and crude mortality outcomes for emergency medical diagnoses and emergency surgical procedures 



Diagnosis/ 
procedure 
Abbreviation 


Acute myocardial 
infarction AMI 


Acute heart 
failure CCF 


Stroke CVA 


Pneumonia 
LRTI 


Pulmonary 

embolism 

PE 


Urosepsis 
UTI 


Sepsis 
SEPSIS 


Cardiac 

arrest 

ARREST 


Acute pancreatitis 
PANC 


Fractured 
neck of 
femur NOF 


Number of coses/ 
procedures 


296673 


206400 


289806 


458064 


74282 


390616 


110240 


13168 


58159 


185567 


Mean patient age 
(years [SE]) 


70.2 (0.026) 


78.4 (0.025) 


75.2 (0.025) 


70.9 (0.027) 


64.4 (0.063) 


69.5 (0.035) 


65.7 (0.061) 


70.0 (0.138) 


56.1 (0.080) 


80.9 (0.026) 


Male proportion 
(% [SE]) 


63.2 (0.0009) 


50.0 (0.0011) 


47.4 (0.0009) 


50.8 (0.0007) 


47.0 (0.0018) 


34.3 (0.0008) 


49.3 (0.0015) 


59.3 (0.0043) 


52.3 (0.0021) 


25.1 (0.0010) 




0(49.2) 


0(0) 


0(0) 


0(32.3) 


0(0) 


0(45.0) 


0(32.1) 


0(37.3) 


0(64.7) 


0(46.5) 


RCS Charlson 

rn-mnrhiHih/ 

score (%) 


1(29.6) 
2(13.3) 


1(32.4) 
2(35.4) 


1(51.2) 
2(31.9) 


1(36.2) 
2(19.5) 


1(58.9) 
2(28.3) 


1(31.2) 
2(15.1) 


1(35.6) 
2(19.9) 


1(32.2) 
2(18.2) 


1(25.1) 
2(7.4) 


1(34.2) 
2(13.4) 




3(7.8) 


3(32.2) 


3(16.8) 


3(12.0) 


3(12.8) 


3(8.7) 


3(12.4) 


3(12.3) 


3(2.8) 


3(5.9) 


Social deprivation 


1(21.4) 


1(22.9) 


1(20.7) 


1(25.1) 


1(19.5) 


1(24.1) 


1(22.8) 


1(21.9) 


1(27.0) 


1(18.5) 


quintile (%) 


2(20.7) 


2(21.4) 


2(20.4) 


2(21.5) 


2(19.7) 


2(21.8) 


2(21.0) 


2(20.6) 


2(21.7) 


2(19.6) 




3(20.7) 


3(20.7) 


3(20.7) 


3(19.7) 


3(20.6) 


3(20.0) 


3(20.0) 


3(20.0) 


3(19.1) 


3(21.1) 




4(19.8) 


4(19.0) 


4(20.3) 


4(18.1) 


4(20.6) 


4(18.4) 


4(19.1) 


4(19.4) 


4(17.3) 


4(21.3) 




5(17.5) 


5(16.1) 


5(18.1) 


5(15.6) 


5(19.6) 


5(15.8) 


5(17.1) 


5(18.1) 


5(15.1) 


5(19.5) 


In-hospital 
mortality 
rate (% [SE]) 


12.1 (0.0006) 


17.0 (0.0008) 


24.8 (0.0008) 


26.3 (0.0007) 


8.0 (0.0010) 


5.1 (0.0004) 


27.7 (0.0013) 


66.7 (0.0041) 


5.7 (0.0010) 


9.8 (0.0007) 


1-year mortality 
rate (% [SE]) 


22.8 (0.0008) 


41.9 (0.0011) 


37.5 (0.0009) 


42.9 (0.0007) 


21.8 (0.0015) 


24.3 (0.0007) 


44.6 (0.0015) 


72.1 (0.0039) 


10.3 (0.0013) 


28.4(0.0010) 


28-day emergency 
readmission rate 
(% [SE]) 


28.6 (0.0008) 


36.9 (0.0011) 


19.0 (0.0007) 


29.2 (0.0007) 


24.2 (0.0015) 


29.4 (0.0007) 


33.2 (0.0014) 


26.4 (0.0038) 


25.1 (0.0018) 


20.0 (0.0009) 


Median (IQR) 
length of stay 
(days) 


6(7) 


7(11) 


11 (24) 


6(9) 


6(6) 


4(9) 


6(11) 


14(21) 


6(7) 


16(18) 


Diagnosis/procedure 


Hernia 
repair 


Appendicectomy 


Upper gastrointestinal 
emergency laparotomy 


Colorectal 

emergency 

surgery 


Emergency 

urological 

surgery 


Ruptured 
abdominal 
aortic 
aneurysm 


Endovascular 
repair of 
ruptured 
AAA 


Emergency carotid 
endarterectomy 


Lower limb 
arterial bypass 


Lower limb 
amputation 


Abbreviation 


HERNIA 


APPEND 


PEPTIC 


COLOLAP 


UROL 


AAA 


EVAR 


CEA 


LEAB 


AMP 


Number of procedures 


42302 


141538 


10237 


42366 


51963 


6687 


729 


1234 


10267 


16411 


Mean patient age 
(years [SE]) 


63.2 (0.091) 


33.9 (0.042) 


59.4(0.195) 


65.5 (0.081) 


59.0 (0.083) 


73.5 (0.102) 


73.5 (0.410) 


71.0 (0.280) 


70.3 (0.141) 


66.5 (0.125) 


Male proportion 
(% [SE]) 


57.0 (0.0024) 


51.7 (0.0013) 


58.5 (0.0049) 


47.5 (0.0024) 


67.0 (0.0021) 


83.3 (0.0046) 


79.4 (0.0150) 


66.5 (0.0134) 


62.9 (0.0048) 


70.2 (0.0036) 




0(65.3) 


0(88.7) 


0(65.7) 


0(35.3) 


0(43.4) 


0(0.5) 


0(3.2) 


0(0.65) 


0(14.1) 


0(12.6) 



Table 1 Demographic data and crude mortality outcomes for emergency medical diagnoses and emergency surgical procedures (Continued) 



RCS Chorlson 


1(24.3) 


1(10.3) 


1(23.5) 


1(42.4) 


1(33.9) 


1(59.7) 


1(44.2) 


1(49.2) 


1(40.0) 


1(27.4) 


co-morbidity 


2(7.5) 


2(0.9) 


2(7.9) 


2(16.1) 


2(15.6) 


2(28.2) 


2(31.1) 


2(31.0) 


2(28.1) 


2(32.4) 


score (%) 


3(3.0) 


3(0.2) 


3(2.9) 


3(6.3) 


3(7.2) 


3(11.6) 


3(21.5) 


3(19.2) 


3(17.8) 


3(27.6) 




1(21.2) 


1(20.8) 


1(31.3) 


1(19.3) 


1(20.6) 


1(17.6) 


1(19.4) 


1(18.0) 


1(25.8) 


1(25.6) 


Social deprivation 
quintile (%) 


2(20.9) 
3(20.8) 
4(19.7) 


2(20.7) 
3(20.0) 
4(19.4) 


2(21.7) 
3(17.6) 
4(16.2) 


2(19.7) 
3(20.6) 
4(20.7) 


2(20.1) 
3(20.2) 
4(19.7) 


2(20.1) 
3(22.0) 
4(21.6) 


2(20.4) 
3(19.3) 
4(22.5) 


2(20.7) 
3(21.0) 
4(22.0) 


2(21.4) 
3(20.4) 
4(17.4) 


2(22.3) 
3(20.3) 
4(17.6) 




5(17.4) 


5(19.2) 


5(13.2) 


5(19.8) 


5(19.5) 


5(18.6) 


5(18.5) 


5(18.4) 


5(15.1) 


5(14.2) 


In-hospital 
mortality 
rate (96 [SE]) 


3.1 (0.0008) 


0.2 (0.0001) 


16.7 (0.0037) 


15.7 (0.0018) 


3.6 (0.0008) 


33.5 (0.0058) 


13.2 (0.0125) 


1.4 (0.0033) 


12.1 (0.0032) 


10.5 (0.0024) 


1-year mortality 
rate (% [SE]) 


8.7 (0.0014) 


0.5 (0.0002) 


22.1 (0.0041) 


29.6 (0.0022) 


20.7 (0.0018) 


38.4 (0.0059) 


22.8 (0.0155) 


6.8 (0.0072) 


26.5 (0.0044) 


25.8 (0.0034) 


28-day emergency 
readmission rate 
(% [SE]) 


15.7 (0.0018) 


10.8 (0.0008) 


16.3 (0.0037) 


23.1 (0.0020) 


29.4 (0.0020) 


22.3 (0.0051) 


31.3 (0.0172) 


14.2 (0.0010) 


32.8 (0.0046) 


25.8 (0.0034) 


Median (IQR) 
length of stay 
(days) 


3(4) 


3(2) 


9(11) 


15(15) 


5(8) 


16(17) 


10(11) 


5(6) 


14(18) 


20 (31) 



SE = Standard Error. 
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Table 2 Proportions of hospitals with risk-adjusted outcomes for each of the conditions/procedures lying above or 
below the 95% control limits 



Clinical In-hospital mortality One-year mortality 28-day emergency readmission 

group 





Proportion of 

llU3|JllCll3 UclUW 

lower 95% control 
limit 


Proportion of 

h/~»cr\it;»lc akm\/a 
1 ILofJI lal:> dUUVc 

upper 95% control 
limit 


Proportion of 

llU3|Jllcll3 UclUW 

lower 95% control 
limit 


Proportion of 

I'm c nit ale ahn\ra 
llU3|Jllcll3 dUUVc 

upper 95% control 
limit 


Proportion of 

1 ILojJI lal:> Uciuw 

lower 95% control 
limit 


Proportion of 

llCopilcilo dUUVc 

upper 95% control 
limit 


AMI 


16.3% 


21.7% 


10.2% 


1 7.5% 


7.9% 


12.1% 


CCF 


1 7.8% 


20.7% 


9.8% 


16.1% 


4.1% 


9.3% 


CVA 


19.9% 


22.7% 


12.2% 


19.3% 


10.3% 


15.4% 


LRTI 


22.2% 


28.1% 


16.8% 


25.4% 


7.8% 


14.5% 


PE 


10.8% 


14.5% 


3.0% 


6.6% 


3.6% 


6.0% 


UTI 


14.1% 


1 7.7% 


9.4% 


16.1% 


12.0% 


18.2% 


SEPSIS 


1 7.7% 


23.2% 


9.9% 


10.5% 


3.9% 


6.2% 


ARREST 


1 .8% 


2.4% 


1 .2% 


1 .2% 


0% 


5.6% 


PANC 


1 .2% 


4.9% 


3.7% 


4.3% 


4.3% 


1 .8% 


NOF 


18.1% 


19.4% 


8.1% 


14.4% 


1 1 .3% 


14.4% 


HERNIA 


2.5% 


6.7% 


1 .8% 


3.7% 


1 .2% 


3.1% 


APPEND 


0.6% 


3.1% 


1 .3% 


5.0% 


12.6% 


15.1% 


PEPTIC 


1 .9% 


1 .2% 


0.6% 


2.5% 


0.6% 


4.4% 


COLOLAP 


5.5% 


10.4% 


1 .2% 


4.3% 


2.4% 


3.7% 


UROL 


3.0% 


9.5% 


4.7% 


7.1% 


4.7% 


7.7% 


AAA 


5.8% 


7.2% 


4.3% 


5.0% 


1 .5% 


4.5% 


EVAR 


0% 


2.4% 


0% 


2.4% 


0% 


2.5% 


CEA 


0% 


0% 


0% 


3.8% 


0% 


6.6% 


LEAB 


0% 


2.8% 


0% 


4.2% 


0.7% 


3.6% 


AMP 


2.4% 


8.5% 


1 .2% 


4.2% 


0.6% 


4.2% 



Outcomes shown are risk-standardised in-hospital mortality, one-year mortality and 28-day emergency readmission rates. 



In-hospital all-cause risk-standardised mortality rate 
(in-RSMR) 

For in-RSMR, the clinical groups with the greatest pro- 
portion of hospitals lying outside statistical control 
limits were medical diagnoses, including acute myocar- 
dial infarction, heart failure, stroke, pneumonia and 
sepsis. For each of these, more than 20% of hospitals 
lay above the 95% confidence limit. This was in addition 
to wide variations in the actual in-RSMR for each diagno- 
sis, with many diagnoses demonstrating more than doub- 
ling of mortality rates between hospitals. An illustrative 
funnel plot for fractured neck of femur is given in 
Figure 1. 

Within surgical diagnoses, significant variations in 
mortality were seen for fractured neck of femur, emer- 
gency colorectal laparotomy, ruptured abdominal aortic 
aneurysm repair, and emergency urological interventions. 
Of particular note, for fractured neck of femur, 19.4% of 
hospitals lay above the upper 95% confidence limit for 
in-RSMR. 



One-year all-cause risk-standardised mortality rate (1 yr-RSMR) 

A number of medical conditions displayed wide varia- 
tions in 1 yr-RSMR. For acute myocardial infarction, 
heart failure, stroke, pneumonia and urinary tract infec- 
tion more than 15% of hospitals lay above the upper 
95% confidence limit. An illustrative funnel plot for acute 
myocardial infarction is given in Figure 2. 

Mortality variations after emergency surgery were less 
pronounced for 1 yr-RSMR than for in-RSMR. Signifi- 
cant variations were still observed, most notably, for 
fractured neck of femur where 14.4% of hospitals lay 
above the upper 95% confidence limit for 1 yr-RSMR. 

28-day all-cause risk-standardised emergency readmission 
rate (RSRR) 

The widest variations in RSRR were for appendicectomy, 
fractured neck of femur, urinary tract infections, pneumo- 
nia, stroke and acute myocardial infarction with over 10% 
of hospitals lying above the upper 95% confidence limit. An 
illustrative funnel plot for pneumonia is given in Figure 3. 
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NOF (excludes trusts with <10 expected events) 
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Figure 1 Example of a funnel plot visually exploring the issue of variability in all-cause in-hospital mortality rate for fractured neck of 
femur (NOF) between providers. 37.5% of hospitals lay outside the 95% (green lines) confidence intervals (green) and 18.1% of hospitals lay 
outside the 99.8% (red lines) confidence intervals. Further statistical analysis determined whether the probability that the number of hospitals 
outside the confidence intervals was significant (both p < 0.0001). 



Within provider inter-relationship in outcome between 
clinical groups 

First, significant differences were observed between aggre- 
gated outcome quintiles when considering all 20 groups 
together for in-RSMR, 1 yr-RSMR and RSRR (p < 0.0001 



for each). This confirmed hospital-wide variations in the 
overall outcomes delivered by individual hospitals for 
acute medical and surgical admissions. 

Second, the outcomes for specific patient groups within 
a hospital were strongly associated with the aggregate 



AMI (excludes trusts with <10 expected events) 
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Figure 2 Example of a funnel plot visually exploring the issue of variability in all-cause 1-year mortality rate for acute myocardial infarction 
(AMI) between providers. 27.7% of hospitals lay outside the 95% (green lines) confidence intervals (green) and 1 1.4% of hospitals lay outside the 99.8% 
(red lines) confidence intervals. Further statistical analysis determined whether the probability that the number of hospitals outside the confidence 
intervals was significant (both p < 0.0001). 
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Pneumonia (excludes trusts with <10 expected events) 




expected 

Figure 3 Example of a funnel plot visually exploring the issue of variability in all-cause 28-day emergency readmission rate for 
pneumonia (LRTI) between providers. 22.3% of hospitals lay outside the 95% (green lines) confidence intervals (green) and 8.9% of 
hospitals lay outside the 99.8% (red lines) confidence intervals. Further statistical analysis determined whether the probability that the 
number of hospitals outside the confidence intervals was significant (both p < 0.0001). 



outcome of the other clinical groups within the same hos- 
pital (Additional file 1: Tables S3-S5 and Figures S1-S3). 
For those hospitals with the lowest aggregate in-RSMR 
(quintile 1), the in-RSMR in most cases was also lowest 
for each clinical group being tested. Using a more specific 
example, for acute heart failure, hospitals in aggregate 
quintile 1 had a in-RSMR of 15.0% whereas hospitals in 
aggregate quintile 5 had a in-RSMR of 19.8% (Figure 4). 
The mortality rate increased sequentially from quintile 1 
to 5 demonstrating that there was a significant relation- 
ship between the aggregate in-RSMR and the heart- 
failure specific outcome (p < 0.001). This result was 
maintained for 1 yr-RSMR with 40.1% mortality in 
quintile 1 and 44.6% in quintile 5 and a progressive in- 
crease between quintiles (p < 0.001; Figure 5). 

This was also seen in surgical procedures, such as frac- 
tured neck of femur in which quintile 1 had an in- 
RSMR of 7.85% and 11.3% in quintile 5, a 43.9% increase 
in risk across quintiles. The aggregate outcome was 
strongly associated with fractured neck of femur-specific 
outcome (p < 0.001). At one-year, the 1 yr-RSMR was 
27.0% for quintile 1 and increased across quintiles to 
31.0% for quintile 5 (p < 0.001). 

Diagnoses such as sepsis showed similar results with 
the best outcomes being in quintile 1 for in-RSMR 
24.3% rising to 32.4% in quintile 5. This 33.3% risk in- 
crease was incremental across quintiles, and demon- 
strated a strong association with the sepsis-specific outcome 



(p < 0.001). Similarly, the 1 yr-RSMR for sepsis was 41.1% for 
quintile 1 rising to 48.7% for quintile 5 (p < 0.001). 

Even for surgical procedures with low event rates the 
relationships were notable. For example, a doubling of 
in-RSMR was observed for appendicectomy between 
quintile 1 (0.119%) and quintile 5 (0.252%), but the aggre- 
gate outcome was associated with the appendicectomy- 
specific outcome for in-RSMR (p = 0.02) and 1 yr-RSMR 
(quintile 1 = 0.387%, quintile 5 = 0.574%; p = 0.001). 

When considering RSRR, the aggregated readmission 
rate was associated with the diagnosis -specific readmis- 
sion rates in many cases. For example, the RSRR after 
fractured neck of femur was 9.80% in quintile 1 rising to 
13.0% for quintile 5 (Figure 6; p < 0.001). 

Discussion 

This study analysed nearly 2.5 million emergency admissions 
to acute English NHS Trusts over a contemporary five-year 
period, with one year follow-up. The risk-standardised ana- 
lyses covered the commonest acute medical and acute surgi- 
cal diagnoses and procedures using granular, and clinically 
meaningful subgroups. 

The key finding of this study was to demonstrate signifi- 
cant variations between hospitals in terms of the in-hospital 
mortality rates, one-year mortality rates and 28-day emer- 
gency readmission rates that followed non-elective medical 
or surgical admissions. Furthermore, highly relevant 
within-provider inter-relationships were seen such that 
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Figure 4 Example of the inter-relationship analysis for heart failure (all-cause in-hospital mortality rate). 



the outcome of a particular condition in one hospital 
was strongly associated with the aggregated outcome 
of the other conditions in that hospital 

These data are suggestive of the presence of hospital- 
level factors that determine the outcomes of a variety of 
disparate emergency conditions [25,26]. In some cases, 



this translates into the outcomes from the same diagno- 
sis being significantly better in one hospital than in an- 
other hospital. Importantly, in addition to this inter- 
hospital variation, the outcomes of apparently disparate 
diagnoses within individual providers are inter-related, 
such that systemic structural and process factors are 
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Figure 5 Example of the inter-relationship analysis for heart failure (all-cause 1-year mortality rate). 
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Quintile of "aggregate" 28-day risk-adjusted emergency readmission (1 = lowest, 5 = highest) 
Figure 6 Example of the inter-relationship analysis for fractured neck of femur (all-cause 28-day emergency readmission). 



implicated that underlie clinical outcomes. These factors 
remain incompletely defined, but appear to contribute to 
a system-wide phenomenon that persisted in analyses of 
28-day emergency readmission rates and one-year mor- 
tality. Whilst it is likely that some of these influences are 
definable structure or process factors, they might also be 
representative of more abstract concepts such as institu- 
tional attitudes that stimulate excellence [27,28]. The 
scale of variation between hospital outcomes was con- 
cerning, and often exceeded both statistical and clinical 
boundaries of acceptability. At a time when the quality 
of care delivered is being closely examined, such varia- 
tions require further investigation. Modifiable technical, 
organisational or hospital-related factors play an import- 
ant role in patient care, and merit further study in order 
to optimise service delivery and to improve care. Hospital- 
level factors which have thus far been demonstrated to 
influence outcomes for certain defined clinical groups 
include availability of high-dependency beds with ad- 
equate medical staffing levels, co-localisation of ancil- 
lary specialties, nurse-to-patient ratios, teaching status 
and hospital size [29-33]. Process factors are more het- 
erogeneous and consequently can be more difficult to 
identify but have, nonetheless, also been shown to in- 
fluence outcomes [34-36]. This supposition is further 
supported by the findings of others that the association 
between hospital procedural volume and post-operative 



mortality lacks specificity [9]. Good governance practices 
such as complete and accurate data collection/submission, 
internal audit, transparent publication of results and bench- 
marking against defined quality standards perpetuate good 
performance. Gauging improvement against national Qual- 
ity Improvement Frameworks requires objective appraisal 
of outcomes and must be accompanied by a willingness to 
change sub-standard practices and to embrace service 
remodelling if necessary. Robust local monitoring and 
mandatory submission of data is needed to ensure 
early reaction to divergent performance and will be of 
interest to commissioners who may use evidence of 
such practice to shape service delivery [37]. Transpar- 
ent reporting of results with clear elaboration of statis- 
tical methodology used to risk-adjust data and identify 
outliers and peer-review of outcomes in an environ- 
ment seeking to improve quality of care rather than to 
ostracize individuals or units is essential [14]. Interest- 
ingly, current prevailing opinion is that supposed patient 
empowerment through reporting of clinical outcomes in 
their current form is unlikely to drive service improve- 
ment through market forces as patients tend rely more on 
subjective information or patient experience measures 
(PEMs) such as hospital cleanliness [38]. 

On the basis of these results, the development of an 
outcomes rating system could be cautiously supported, 
as this work has shown that there is a need for improved 
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outcomes and reduced variations in outcome in emer- 
gency care [39-41]. However, in the first instance, any 
composite rating system must be based on clinically 
meaningful subgroups, with definable endpoints. A sin- 
gle pan-provider rating is unlikely to be the correct 
model [42-44]. These results suggest that acute condi- 
tions require detailed appraisal at the speciality level at 
least. Furthermore, a single rating system that purports 
to cover both hospitals' clinical and non-clinical per- 
formance will likely be misleading [3]. 

Strengths and limitations 

Clinical outcomes in this study were limited to validated 
hard endpoints. As a contemporary five-year cohort was 
used, concerns about the quality of coding in adminis- 
trative data are mitigated as systematic reviews have 
found acceptable coding accuracy rates within the HES 
data, with data quality improving in more recent years 
[45,46]. Uniquely, HES data allows for linkage to out-of- 
hospital death - a powerful feature which adds robust- 
ness to the findings through the use of 1-year mortality 
rates. In addition, recognised and published techniques 
were used to risk-standardise the data, and to determine 
the presence and strength of inter- relationships [8,17]. 

Another strength of the study lies in case selection 
that focussed on a group of commonly encountered 
medical and surgical pathologies and ensured a plausible 
link between mortality and quality by including patients 
with conditions amenable to salvage [47]. Using the ex- 
ample of colorectal cancer as an example - as the 
COLOLAP group included only patients who underwent 
surgery - the majority of patients with disseminated dis- 
ease requiring only palliation would have been excluded. 
Similarly the AAA group included only patients who 
underwent surgery and thus would have excluded mori- 
bund patients judged too unfit for surgery. This supports 
findings that restricting the calculation of standardised 
mortality ratios to include only certain conditions re- 
duces over-dispersion of data and may yield a more useful 
comparative statistic [48]. This clinically meaningful case 
selection addresses the criticism of risk-standardised out- 
comes that there is a poor correlation between quality of 
care provided and probability of death [42,49]. Until 
complete and accurate disease-staged outcomes data are 
available on the same scale as current administrative data, 
risk-standardised outcomes remain the best measure of 
clinical outcomes for national studies. Whilst data from 
clinical databases and registries may seem more attractive 
than HES data, their utility remains hampered by the lack 
of mandatory submission from all providers [39,50] . 

Finally, the use of a super-spell definition of in-hospital 
mortality and the inclusion of 1-year mortality and 28-day 
emergency readmissions as secondary outcome measures 
demonstrated that the findings were consistent and 



mitigated criticisms of in-hospital mortality as an out- 
come measure, namely that it can be confounded by 
institutional, social and financial factors favouring 
rapid discharge [51,52]. Despite criticisms of in-hospital 
mortality as an outcome metric, it should be noted that 
studies indicate that its value for internal benchmarking 
purposes is comparable to other mortality outcomes 
whilst others have demonstrated that different mortal- 
ity metrics are themselves highly correlated at hospital 
level [7,53]. With particular regard to health outcomes 
research using HES data, in-hospital mortality remains 
the most commonly used metric [12]. 

Limitations of this study include the possibility of 
inter-hospital coding variability inherent to the use of 
retrospective administrative data and it is acknowledged 
that there remains no universal consensus on risk ad- 
justment methodology with the consequence that differ- 
ent models can yield conflicting results [43,47]. Whilst it 
is not possible to definitively state that the hospital to 
hospital variations observed are not at least partially a 
result of inadequate adjustment for case-mix rather than 
reflecting genuine differences in quality of care, it should 
be noted that, with regard to HES data, the covariates 
used for risk adjustment have been shown to produce 
regression models of comparable discriminatory power 
to those derived from clinical databases for colorectal 
and vascular surgery [7,18,54,55]. Additionally, it is ac- 
knowledged that geographic variations in primary care 
and social care facilities are not included in the model 
and as such there may be unmeasured confounding 
given the effect that these factors can have on mortality 
and readmission rates. Interestingly however, none of 
the standardised funnel plots showed evidence of over- 
dispersion of data suggesting that the influence of un- 
controlled factors in the risk- adjustment process was not 
significant [14,56]. 

The RCS Charlson score has been specifically modified 
and validated against the HES data [19]. Nonetheless, 
criticism has been directed towards the use of the Charlson 
co-morbidity index for case mix adjustment as the index 
is itself a function of coding depth and accuracy and 
consequently displays non-constant risk relationships 
amongst hospitals [43]. In the present study, a post-hoc 
sensitivity analysis performed by excluding the RCS 
Charlson score as a regression covariate did not alter 
the present findings (data not shown). This finding is 
consistent with research suggesting that exclusion of 
Charlson scores when calculating RSMRs has only a 
modest effect and that co-morbidity recording in HES 
data is not associated with widespread bias [48]. The 
use of the RCS version of the Charlson score in the ana- 
lyses acknowledges the finding that modifications of the 
original Charlson co-morbidity index (i.e. that originally 
described for US administrative data) to take account of 
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local coding practices can improve the discriminatory 
power of logistic regression models [57]. 

The use of mortality as a metric for assessing the qual- 
ity of care has been criticised although it remains a com- 
monly used measure in health outcomes research and is 
particularly relevant in the context of emergency admis- 
sions [12,14,42]. 

It is acknowledged that the study does not address 
which critical structural and process measures account 
for the observed inter-relationships and that the results 
are not necessarily generalisable to paediatric popula- 
tions, elective care admissions or to healthcare systems 
outside the United Kingdom. Determination of the under- 
lying structures and processes of care remains the focus of 
ongoing research. 

Conclusions 

For emergency medical and surgical admissions in England, 
wide variations in outcome exist between providers. In 
addition, strong associations in outcome were found be- 
tween disparate clinical groups within individual providers 
that suggested the presence of underlying global structure 
and process factors underpinning clinical outcomes. 
These results have implications for the way in which 
care is delivered and provides potential targets for glo- 
bal quality improvement. 
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